---
id: llm-custom
sidebar_label: Customizing LLM Components
title: Customizing LLM based Components
abstract:
---

import RasaProLabel from "@theme/RasaProLabel";
import RasaLabsLabel from "@theme/RasaLabsLabel";
import RasaLabsBanner from "@theme/RasaLabsBanner";

<RasaProLabel />

<RasaLabsLabel />

<RasaLabsBanner version="3.7.0b1" />

The LLM components can be extended and modified with custom versions. This
allows you to customize the behavior of the LLM components to your needs and
experiment with different algorithms.

## Customizing a component

The LLM components are implemented as a set of classes that can be extended
and modified. The following example shows how to extend the 
`LLMIntentClassifier` component to add a custom behavior.

For example, we can change the logic that selects the intent labels that are 
included in the prompt to the LLM model. By default, we only include a selection
of the available intents in the prompt. But we can also include all available
intents in the prompt. This can be done by extending the `LLMIntentClassifier`
class and overriding the `select_intent_examples` method:

```python
from rasa_plus.ml import LLMIntentClassifier

class CustomLLMIntentClassifier(LLMIntentClassifier):
    def select_intent_examples(
        self, message: Message, few_shot_examples: List[Document]
    ) -> List[str]:
        """Selects the intent examples to use for the LLM training.

        Args:
            message: The message to classify.
            few_shot_examples: The few shot examples to use for the LLM training.

        Returns:
            The list of intent examples to use for the LLM training.
        """
        
        # use all available intents for the LLM prompt
        return list(self.available_intents)
```

The custom component can then be used in the Rasa configuration file:

```yaml title="config.yml"
pipeline:
  - name: CustomLLMIntentClassifier
    # ...
```

To reference a component in the Rasa configuration file, you need to use the
full name of the component class. The full name of the component class is
`<module>.<class>`.

All components are well documented in their source code. The code can 
be found in your local installation of the `rasa_plus` python package. 

## Common functions to be overridden
Below is a list of functions that could be overwritten to customize the LLM
components:

### LLMIntentClassifier

#### select_intent_examples

Selects the intent examples to use for the LLM prompt. The selected intent 
labels are included in the generation prompt. By default, only the intent
labels that are used in the few shot examples are included in the prompt.

```python
    def select_intent_examples(
        self, message: Message, few_shot_examples: List[Document]
    ) -> List[str]:
        """Returns the intents that are used in the classification prompt.

        The intents are included in the prompt to help the LLM to generate the
        correct intent. The selected intents can be based on the message or on
        the few shot examples which are also included in the prompt.

        Including all intents can lead to a very long prompt which will lead
        to higher costs and longer response times. In addition, the LLM might
        not be able to generate the correct intent if there are too many intents
        in the prompt as we can't include an example for every intent. The
        classification would in this case just be based on the intent name.

        Args:
            message: The message to classify.
            few_shot_examples: The few shot examples that can be used in the prompt.


        Returns:
        The intents that are used in the classification prompt.
        """
```

#### closest_intent_from_training_data
The LLM generates an intent label which 
might not always be part of the domain. This function can be used to map the
generated intent label to an intent label that is part of the domain.

The default implementation embedds the generated intent label and all intent
labels from the domain and returns the closest intent label from the domain.

```python
    def closest_intent_from_training_data(self, generated_intent: str) -> Optional[str]:
        """Returns the closest intent from the training data.

        Args:
            generated_intent: the intent that was generated by the LLM

        Returns:
            the closest intent from the training data.
        """
```

#### select_few_shot_examples

Selects the NLU training examples that are included in the LLM prompt. The
selected examples are included in the prompt to help the LLM to generate the
correct intent. By default, the most similar training examples are selected. 
The selection is based on the message that should be classified. The most
similar examples are selected by embedding the incoming message, all training
examples and doing a similarity search.

```python
    def select_few_shot_examples(self, message: Message) -> List[Document]:
        """Selects the few shot examples that should be used for the LLM prompt.

        The examples are included in the classification prompt to help the LLM
        to generate the correct intent. Since only a few examples are included
        in the prompt, we need to select the most relevant ones.

        Args:
            message: the message to find the closest examples for

        Returns:
            the closest examples from the embedded training data
        """
```

### LLMResponseRephraser

#### rephrase

Rephrases the response generated by the LLM. The default implementation
rephrases the response by prompting an LLM to generate a response based on the
incoming message and the generated response. The generated response is then
replaced with the generated response.

```python
    def rephrase(
        self,
        response: Dict[str, Any],
        tracker: DialogueStateTracker,
    ) -> Dict[str, Any]:
        """Predicts a variation of the response.

        Args:
            response: The response to rephrase.
            tracker: The tracker to use for the prediction.
            model_name: The name of the model to use for the prediction.

        Returns:
            The response with the rephrased text.
        """
```

### IntentlessPolicy

#### select_response_examples

Samples responses that fit the current conversation. The default implementation
samples responses from the domain that fit the current conversation.
The selection is based on the conversation history, the history will be 
embedded and the most similar responses will be selected.

```python
    def select_response_examples(
        self,
        history: str,
        number_of_samples: int,
        max_number_of_tokens: int,
    ) -> List[str]:
        """Samples responses that fit the current conversation.

        Args:
            history: The conversation history.
            policy_model: The policy model.
            number_of_samples: The number of samples to return.
            max_number_of_tokens: Maximum number of tokens for responses.

        Returns:
            The sampled conversation in order of score decrease.
        """
```

#### select_few_shot_conversations

Samples conversations from the training data. The default implementation
samples conversations from the training data that fit the current conversation.
The selection is based on the conversation history, the history will be
embedded and the most similar conversations will be selected.

```python
    def select_few_shot_conversations(
        self,
        history: str,
        number_of_samples: int,
        max_number_of_tokens: int,
    ) -> List[str]:
        """Samples conversations from the given conversation samples.

        Excludes conversations without AI replies

        Args:
            history: The conversation history.
            number_of_samples: The number of samples to return.
            max_number_of_tokens: Maximum number of tokens for conversations.

        Returns:
            The sampled conversation ordered by similarity decrease.
        """
```